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(57) A method of reducing the dynamic range of an 
input image for effectively rendering the input image on 
an output display medium, the method comprises the 
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from the input image; generating contrast gain-control 
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signals from the input image by detecting coarse scale 
edges; modifying the detail signals according to the 
contrast gain signals; and, adding the modified detail 
signals to the coarse signals to obtain an output image. 
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Description 

FIELD OF THE INVENTION 

? [0001] This invention relates to an automatic tone adjustment method for computer processing of digital images which 
have a dynamic range larger than the intended output display media, such as photographic paper, monitor display, and 
transparent film. 

BACKGROUND OF THE INVENTION 

10 

[0002] Many image capture devices can record a signal dynamic range that is wider than a typical display device can 
render effectively. For example, consumer color negative films can record scene luminance range of 1000:1, but photo- 
graphic color papers can only render a luminance range from 50:1 to 100:1 . Another example is the storage phosphor 
used in digital radiography, which can capture an x-ray exposure dynamic range of 10,000:1 , but a radiographic film as 
is an output medium can render only a small fraction of it (about 20:1) in good contrast. In both examples, wide dynamic 
range images will suffer very significant loss of details in the highlight and the shadow, when they are displayed on their 
intended output media. Several inventions have been disclosed for solving such problems. 

[0003] The best known method is the unsharp masking and its variations. U.S. Patent No. 5,012,333 and U.S. Patent 
No. 4,812,903 teach methods of decomposing an input image into a low-frequency component and a high-frequency 

20 component- By modifying the low-frequency component with a nonlinear tone scale curve and amplifying the high-fre- 
quency component (unsharp masking), one can combine the two modified components back to form a new image 
which has a smaller dynamic range and yet retain most image details. U.S. Patent No. 5,608,813 and U.S. Patent No. 
5,454,044 teach a similar method, but formulates it in spatial domain. The low-frequency (unsharp) component is 
obtained by averaging image values over a local window. The low-frequency component is then used in a monotonically 

?5 decreasing function to determine how much gray value is to be added or substracted from the original image. In 
essence, the method used the unsharp version of the original image to mask off the dynamic range in the input signal. 
These three methods are mathematically similar to the well known unsharp masking technique in photography, and all 
of them suffer from the same drawback of creating dark and bright banding around high-contrast edges as the unsharp 
masking techniqure does. The cause of this artifact is illustrated in Figure 1 . The unsharp (low-frequency) component 

30 rounds off a sharp edge signal to produce a smoothed version. The difference signal (the high-frequency component) 
between the original image and the unsharp version thus contains an overshoot and an undershoot around a sharp 
edge. When the difference signal (the high-frequency) is added back to the dynamic-range compressed low-frequency 
component, the uncompressed or amplified overshoot and undershoot signals around a sharp edge will create a bright 
band and a dark band near the edge. This artifact is more visible near high contrast edges. This artifact is called the 

35 edge banding artifact. The e dge banding artifact is a very ob jec tionable artifact in consum er image *; and a p ntpntiatly 
darigerou§_ arttfact in digital radiography, in order to deal wrtn tms banging problem, U.S. Patent 5,467,404 teaches a 
meffuxf of decomposing an image into multiple resolutions and using a predetermined nonlinear amplitude compres- 
sion function for the high frequency component in each resolution. If the high frequency amplitude is small, it is passed 
unchanged or even amplified. If the high frequency amplitude is large, then it is attenuated. The assumption is that 

to whenever the signal amplitude gets large, regardless of its cause, one should reduce its amplitude so that the overshoot 
and the undershoot near a sharp edge can be properly suppressed. A major deficiency of the method is that the ampli- 
tude in the each resolution alone can not be used to adequately identify whether the large amplitude is caused by a high 
contrast edge or an image texture. Without making the distinction, fine details and textures of high amplitudes will also 
be suppressed, resulting loss in details. A second major deficiency is that a predetermined nonlinear amplitude com- 

*s pression can not adaptively adjust the right amount of compression needed for edges in each resolution of image 
details at different spatial locations. Since every input image has a different dynamic range and thus requires varying 
amount of dynamic range compression. The amount of compression has to be coupled with how the algorithm adjusts 
the amplitude compression for each high frequency component. This need was mentioned in U.S. Patent No. 
5,493,622, but it did not teach how to compute the necessary tone compression curve, nor did it teach how the tone 

so compression curve should be used to deal with the edge banding artifacts. 

[0004] A third deficiency is that the frequency decomposition as described in U.S. Patent 5,467,404 has to sum up 
pixelwise (after interpolation to the required number of pixels) to the original image when no modification is performed 
on any of the signal amplitudes in the decomposition. This constraint limits the types of detailed signals that can be 
extracted from the decomposition. 

55 [0005] Another related approach to the frequency decomposition and modification method is the wavelet transform 
and inverse transform. For example, J. Lu, D.M. Healy, Jr., and J.B. Weaver disclosed such a method in the paper "Con- 
trast enhancement of medical images using multiscale edge representation," Optical Engineering, 33, 7, 2151-2161, 
1994. 
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[0006] A similar invention was also disclosed in US. Patent 5,71 7,791 . In these two prior arts, the goal was to adjust 
local image contrast through a multiresolution analysis. Although the multiscale edge structure used by both prior arts 
is indeed flexible and its proper use allows one to alleviate the third deficiency mentioned above, both prior arts still suf- 
fer the first two major deficiencies pointed out earlier: there was no ex plicit interaction between edges from different 

5 spatial sea lesand n o explicit method to compute how much contrast should be ad justed. Both problems are solved in 
Tms new invention. A contrast gain-control (CGC) module in this new invention derives gain factors from edges of all 
scales and apply them to control how much edge information to add back to the image information at each scale. A tone 
scale curve (TSC) module in this new invention explicitly maps the input dynamic range to the dynamic range of the out- 
put medium. The details of these will be explained shortly in the following sections. 

io [0007] A different idea to deal with the edge banding artifact is disclosed in U.S. Patent No. 5,471 ,987 and U.S. Patent 
No. 5.786,870. where the unsharp (low frequency) component is calculated from a weighted average of a predeter- 
mined mask. The weighting is chosen so that less weight is applied to a given peripheral pixel in the mask if its absolute 
difference from the center pixel is large. The weighting is also chosen so that less weight is applied to a peripheral pixel 
if it is located farther away from the center pixel. By such a differential weighting scheme, pixels across a high contrast 

is edge are not avei aged together and therefore, the unsharp component has relatively sharp high contrast edges. If the 
image detail (high frequency) component is calculated by subtracting the unsharp component from the original image 
signal, then the image detail (high frequency) component will not have the overshoots or undershoots near a high con- 
trast edge. Therefore, there will be no edge banding artifacts when the unsharp component and the image detail com- 
ponent are combined. This method is fairly effective. However, there are two major drawbacks in such a method. The 

20 first drawback is that it is very slow, because the weighted averaging operation is not a regular convolution operation. 
The second drawback is that the method can only deal with hj gh^contrast edges that have a short tra pfi i tirtn m^tan^o |f 
a luminance transition occurs continously over some longer distance, the absolute difference between the center pixel 
and its neighboring pixels will not be large, and therefore the weighting will be relatively uniform over the transition. The 
unsharp component will then be a blurred version of the original image and the image detail component will contain 

25 overshoots and undershoots which, when added back or enhanced, will be visible in the output image. 



SUMMARY OF THE INVENTION 



[0008] The present invention is directed to overcoming one or more of the problems set forth above. Briefly summa- 
30 rized, according to one aspect of the present invention, the invention resides in a method of reducing the dynamic range 
of an input image for effectively rendering the input image on an output display medium, the method comprising the 
steps of (a) extracting detail signals and coarse signals from the input image; (b) generating contrast gain-control sig- 
nals from the input image by detecting coarse scale edges; (c) modifying the detail signals according to the contrast 
gain signals; and, (d) adding the modified detail signals to the coarse signals to obtain an output image. 
35 [0009] An object of this invention is to provide an effective means for compressing the dynamic range of an input &o?fOf 
image so that it can be properly rendered to an output display medium that has a narrower dynamic range, without the y 
deficiencies of the prior arts as mentioned above. > ' ^firC/f^ 

[0010] Another object of this invention is to provide a means for automatically computing an image-dependent tone 
scale (gradation) curve for dynamic range compression. Still, another object of the invention is to provide a means for 
40 corrputing, gain-contr ol signals for suppressing the edge banding artifacts arourx^hrgrvcom eda5^ . Yet another 
object is to provide a general and flexible image decomposition that allows explicit extraction of im^rtSrrtimage details, 
such as edges, lines, and textures. 



BRIEF DESCRIPTION OF THE DRAWINGS 

45 

[0011] 



Figure 1 shows how the banding artifact is generated by prior arts; 

Figure 2 shows one embodiment of this invention. The output image is reconstructed from the weighted average of 
so the decomposition fitter outputs from the original image and its tone scaled version; 

Figure 3 is a block diagram of the preferred embodiment of this invention. It shows the three major modules that 

function together to produce a dynamic range compressed output image l out ; 

Figure 4 is an expansion of Figure 3 with more details of the filter banks module; 

Figure 5 is a block diagram of the tone scale curve (TSC) module; 
55 Figure 6 shows the characteristic curve of an example photographic color paper. The diagram also shows the black 

point, the white point, the black value, and the white value; 

Figure 7 shows the parameters used in the tone scale curve construction; 

Figure 8 shows how an interval of the slope curve is interpolated from the nodes; 
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Figure 9 shows an example of the histogram (the bottom curve) of an input image and the tone scale curve con- 
structed from the histogram; 

Figure 10 shows the ideal way to compress an illumination edge, leaving all the reflectance edges alone; 
Figure 1 1 shows the details of the contrast gain-control module; 
s Figure 12 shows the various low-pass (smoothing) L filters that are used on images of different sizes for computing 

the contrast gain-control signals; 

Figure 13 shows the preferred embodiment of the filter banks (FB) module; and, 

Figure 14 shows a comparison of the input signal profile of a line through the input image / and the output signal 
profile of the same line through the output image l out . 

10 

DETAILED DESCRIPTION OF THE INVENTION 

[0012] An overview of the invention is described with reference to Fig. 2 and Fig. 3. An input image, /, with wide 
dynamic range can be considered as having a component A which contains the global, large-scale dynamic range sig- 

15 nal, and another component B which contains the local, fine scale image details. This can be written as / = A + B . The 
task is to compress the dynamic range of A by applying a compressive tone scale (T) to it. For purpose of enhancing 
the image details in S, an image enhancement operation E to B can be applied. Therefore the processed output image, 
/', can be written as /' = T(A) + E(B) . Unsharp masking in the prior arts uses low-pass filtering or weighted local aver- 
aging to derive the A component from the input image, ard compute B as B = / - A . In this invention, the separation 

20 of the A and B components from / is through a decomposition filter bank. These filters need not be linear. For example, 
since edges are important information in images, the decomposition can be accomplished through edge detectors at 
many spatial scales. On the top half of Fig. 2, the input image is past through the filter bank 10. On the lower half, the 
input image is input through a tone scale processor 40 where it is compressed before it is past through the decomposi- 
tion filter bank 20. If the dynamic range of the input image is small, the tone scale curve will not reduce the dynamic 

25 range further and the output from the upper filter bank will be essentially the same as that from the lower filter bank. If 
the dynamic range of the image is large, then the output from these two filter banks will differ significantly, especially in 
the highlight and shadow regions where the tone scale curve will try to compress. The central question is to decide how 
to best combine the outputs from the two filter banks so that the reconstructed output image will have a narrower 
dynamic range to match the intended output medium, but also have all the useful image details properly rendered so 

30 that they are pleasingly visible. This invention discloses a contrast gain-control means 30 to produce the desired weight- 
ing factors. W and W, to combine the outputs from the two filter banks 1 0 and 20. The output from the decomposition 
filter bank 10 is always weighted more in the low-modulation areas in an image, i.e., W » W. In th e areas where high 
contest edgesoccur, the output from the decomposition filter bank 20 is weig hted more ( W » H^Tand the output image 
is reconstructed^ a reconstruction TlfferTsanirSO. I ne reconstructed oHpuFimage l out will take its high contrast edge 

35 signals from the dynamic range compressed image. The resulting images will have the fine low-contrast details as in 
the original input image and also have the much-reduced high contrast edges as in the dynamic range compressed 
image. 

[001 3] The invention as expressed in Fig. 2 can be performed more efficiently by a different embodiment as shown in 
Fig. 3. Instead of processing the original version of the input image and its tone-scaled version through the decompo- 
se sition filter bank separately, thg jiew embodimen t only proce sses the differe nce between the two versions, ffiereforer 
tJYe^Tcnction-of'^^ 

desired image details or other features that might have been lost inMthe tone scale OTr^res^onja^ 

nw perspective is quite important because the ton^scaled image is considered as the baseline output image that has 

the desired dynamic range and then the algorithm can choose freely what image details is desired to add back to that 

45 baseline output image. This means that the design of the decomposition and the reconstruction filter banks 60 and 70 
has to satisfy fewer constraints than a perfectly analysis-synthesis filter pair has to. The present invention will focus on 
the framework laid out in Fig. 3, with the understanding that Fig. 2 is an alternative embodiment using most of the same 
ideas about the tone scale curve design and the contrast gain-control with between-scale interactions. 
[0014] Still referring to Fig. 3, this invention consists of three major modules: (1) tone scale curve (TSC) module 80, 

so (2) contrast gain-control (CGC) module 90, and (3) filter banks (FB) module 100. The function of the TSC module 80 is 
to generate a tone scale curve that will map the dynamic range of the input image into the dynamic range of the output 
display media. The CGC module 90 is to generate the banding artifact suppression signal to modulate the edge signals 
extracted in the filter banks (FB) module 100. 

[001 5] An input image / is processed through the (TSC) module 80, which computes a tone scale curve for the input 
55 image and maps it through the curve to produce a tone-scaled (dynamic range compressed) image /'. The tone-scaled 
image /' is then subtracted from the input image / to produce a difference image D. The difference image D is proc- 
essed through the filter bank 60, which extracts the edge and detail signals from the difference image O. The edge sig- 
nals are modulated by the contrast gain-control signals from the (CGC) module 90 that uses edge signals from all 
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scales to generate contrast gain-control signals. The output D' from the filter bank 70 therefore contains edges and 
details in the difference image D, with the edge banding artifacts suppressed by the contrast gain-control signals. D' is 
then added to the tone-scaled image /' to produce the output image, l out . 

[0016] The block diagram of the invention in Figure 3 is expanded further in Figure 4, which shows the detailed struc- 
5 ture of the filter banks module 100 (see Fig. 3) and how the gain-control signals are used to modulate the filter bank 
signal processing. The overall image processing steps are described and then they are followed by the detailed descrip- 
tion of each of the three main modules. 

[0017] First, an input image is processed in the tone scale curve (TSC) module 80. The dynamic range of an input 
image, /. is determined from its histogram. A tone scale curve is then constructed to compress the dynamic range of 

io the input image to that of the intended output display media. The input image is then mapped through the tone scale 
curve to produce a toned-scaled image, /'. The difference image O between / and /' (i.e., D = /-/') is then passed 
through a filter bank 60, consisting a set of N decomposition filters, F 1% F 2 . .... and F N 60a, 60b, ... 60n. These filters 
are feature extractors and they can be either linear or nonlinear. Their main function is to extract the image features, 
such as edges, textures, and shadings, that are considered as desirable targets for enhancement or suppression in a 

is given application. The output D 1 through D N from each of the filters is modulated in Modulator M-, through M N (1 10a 
through 1 1 0n) by a gain-control signal which is produced by the contrast gain-control generator 90 from the input image. 
/ and all the outputs from the decomposition filter bank 60. Each of these gain-adjusted signals, D\ through D' N is then 
passed through a reconstruction filter bank 70 consisting of a set of filters H 2> through H N (70a through 70n). The 
outputs from the reconstruction filter bank 70 are then summed together to produce a new difference signal, D\ which 

20 is then added back to the tone-scaled image, /', to produce the desired output image, l out = /' + D\ 

[0018] Images can be captured and digitized by various imaging devices. It is important to calibrate the digitized 
image to the desired metric. In the following description ol this invention, all images are calibrated to relative log scene 
exposures (RLSE), meaning that the log scene exposure has been shifted so that when it is printed digitally, the bright- 
ness of the image looks pleasing. For convenience, the relative log scene exposure is scaled up by a factor of 1 000 so 

25 that one can use only integer values to represent the input and the output images. However, the intermediate results 
from the various filters are represented as floating point numbers, so that proper precision is always retained. 
[001 9] Having described how the overall system works, each module will now be described in detail. 

The Tone Scale Curve (TSC) Module: 

30 

[0020] Figure 5 shows a diagram of the Tone Scale Curve (TSC) Module 80. TTiis module 80 computes a unique tone 
scale curve for each input image and applies it to produce a tone-scaled version of the input image. The major function 
of the tone scale curve is to compress the large dynamic range of the input image into a smaller dynamic range so that 
the output medium can render the image effectively. 

35 [0021 ] The Tone Scale Module 80 uses ^ist&r am accurru ilator^OJ^cpmput e the image histog ramfrom jheinput 
jmaga_Eami^^ the module 130 computes the aim, t he 1 ^! ^!^ and the < br^t_y^u e : of the input image. 

The highlight slcgeTs h , the mid-tone slope S m , and the shadow slope S s are then computed inBlock 140. From these 
computed parameters, a tone scale curve is generated in block 150. The tone scale curve is implemented as a lookup 
table in 160. The input image is mapped through the lookup table 160 to generate the tone-scaled image /'. 

40 [0022] In order to adjust the dynamic range of the input image so that it can be effectively rendered to the intended 
output medium, the dynamic range of the output medium is needed. This information is available once an output 
medium is selected. For example, if the output medium is photographic paper, one can look up the minimum density 
(0 m in) and the maximum density (D max ) of the paper and its dynamic rangejsjijpiXixiqTately D maxJL D min . To account 
for viPwtngfiarp^nH \q a |[p w SfimP r^mMgr^ignnl modulation in the^bfigfrtr^hlig^ slightly 

45 reduced <^ami crange than D max -D min iscohs Erered, For example, Figure 6 shOws-a^ar^eristic^DTvefor a pho- 
tographic paper. Tile ^-axisiui Hie iCgscene exposure and the y-axis is the resulting status A density after the paper is 
processed and developed. In digital printing, the printer can be calibrated to produce a photographic print according to 
such a characteristic curve The minimum density of th e^paper is 0.05 and the maximum density is 2.34. As mentiprtglN 
before, in order to allow for signal modulation, a densigfflOTTgS^^ filficted as the w hite-point on the paper andgTllg^ 

so as the.bfadyDoirrt, resulting in a slightly reduced dynarnirraftgg^MJ^ Because an image carTbe 

prirrt^r^fttierlighter or darker by shifting the log exposure on the x-axis to the rigffiornsTfie left, a density balance point 
needs to be specified where the middle gray in the image shall be printed in order to get a pleasing brightness rendition. 
Thisjensjty balance poin tisi referred to as the aim point of thejm agejhe aim point is to be printed to a paper density 
0.8. The black point ancfffiewhite poinfon thfcry-axis of the papercurve in Figure 6 are mapped to the black value and 

ss the white value on the x-axis. that represents the relative log scene exposure The resulting black value and white value 
are (aim - 800) and {aim + 640). For an input image, a density balance value, i.e. the aim, has to be determined by a 
density balance algorithm, which is typically provided in almost all photofinishing printers. Once this aim value is cho- 
sen, the available dynamic range for the output image is restricted to the range from {aim - 800) to {aim + 640). There- 
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fore, ass*io«yri in Fig. 7, tjiejyailabla^per dynamic range has to be split into the white rarge^R^jaim to aim + 640j Pl) 
and tt ^plad wang e R^(a im to aim - 8$ )0). These two ranges are defined relative to a given displaymedium and they 
are fixed once an output medium is selected. If the input image has pixel values that exceed this range, it is the job of 
the tone scale curve to compress them into this range by adjusting the slopes in the va rious po rtions of the curv e. The 
following describes how the tone scale curve is constructed to accomplish'this dynamic range compression task. 
/ [0023] Fi rst, the TS C module 80 compgjf^tfie Iiistd0r§rir6f '3rf ii^ut image. The upper 99.5 percentile code value of 
the input histogram is tentatively designated as the bright value of the image, and the 0.5 percentile code value, the dark ^ 
value of the image (see Figure 7). The upper range R u of an image is defined to be from the aim of the image to the ^ 
bright value of the image. The lower range R f of an image is from the aim of the image to the dark value of the image. ^ 
These two ranges. R u and R\, vary from image to image, while the other two ranges, R«, a nd R b . arejix ed once an out- 
put medium is chosen. Therefore, the job of the tone scale curve is to xrxgf£R u into RZ, and R t into ^TTr>ere are infi- 
nitely many tone scale curves that can be constructed to do this rna^ingrTfcwever, the following method of 
construction is the preferred embodiment for this invention. 

[0024] Referring to Figures 6 and 7, the tone scale is conducted to satisfy six constraints: (1 ) the dark value is mapped f~ 
to the black value. (2) the slope at the dark value is specified by the shadow slope S 3 ; ( 3) th e aim value of the input 
image is mapped to itself. (4) the slope at the aim value is specified by the mid-tone s{6pe S^Tte) the bright value is 
mapped to the white value; and, (6) the slope at the bright value is specified by the highlignTSlOfSe, S h . 
[0025] Mathematically, this is a problem of interpolation under the constraint that the curve has to pass through 3 
given points at given slopes. Before describing how such a curve can be constructed, how the points and slopes are 
determined from a given image is first specified. First, the lower range R t of the input image is compared with the black 
range, R b , of the output medium. If R f is larger than R b (meaning tone compression is needed in the shadow), then the 
0.5 percentile value of the histogram is taken as the dark value. Otherwise, the input value (aim - 800) is taken as the 
dark value. The shadow slope S s at the dark value in either case is always set at a value larger than 1.0. The preferred 
shadow slope is 2 0 Next, the upper range R u of the input image is compared with the white range, R w , of the output 
medium. If R u is larger than R w (meaning tone compression is needed in the highlight), then the 99 5 percentile value 
of the histogram is taken as the bright value, and the highlight slope S h is empirically set according to the following for- 
mula: 

s *- 1 - 0+ s« 1 "^- < 1 > 

If R u is equal to or smaller than R w , then the input value (aim + 640) is taken as the bright value and the highlight slope 
S h is set to 1 .0. If an algorithm is available for estimating the scene contrast of the input image, the mid-tone slope S m 
can be adjusted accordingly. If such an algorithm is not available, then the preferred value is S m = 1 .0. 
[0026] There are many curves that can pass through three given points (the dark value, the aim. and the bright value) 
at the specified slopes (S s , S m , and S h ). Additional constraints are needed to construct a unique curve. For tone scale 
purpose, the instantaneous slope at any point on the curve is required to be greater than a minimum value S mjn and 
smaller than a maximum value, S max , as specified by the user. This is to ensure that excessive quantization due to too 
high or too low slopes will not occur. It is further required that the slope changes at the three given points have to be 
gentle and the effect of a specified slope decays with distance as a Gaussian function with a specified standard devia- 
tion. 

[0027] The problem is formulated in a more general setting. Given S miru S/nax. points, (x/.y,), / = 1 , 2 n , where x, 

<xj and y, <y y , for all / <y, and n slopes, S min ^ S, z , / = 1, 2 n , a monotonic curve y = f(x) t is needed such 

that y-f = f(Xj), S, = 1(x t ) for / = 1 , 2, n , and S min <> f'(x) z S msx for all x. The problem is solvable only if S min (x y - - 
x,) <, (yj - Yf) ^ S max (xj - x,) for all j > i. This curve construction problem is solved in this invention by a procedure to 
compute r"'(x) first and then the curve y - f(x) is constructed by integrating f (x). The integration is started from the aim 
point and completed in two directions, one going up stopping at the bright value, and the other going down stopping at 
the dark value. In the process of integration, if the local slope at any point is greater than S max , it is set at S max . Hit is 
smaller than S min , it is set at S mjn . 

[0028] To solve the above curve construction problem, two more constraints are imposed so that the solution can be 
efficiently derived: (1) each interval is interpolated independently, and (2) the interpolation function is a Gaussian func- 
tion with a user-specified standard deviation (called width of influence). Figure 8 shows how f'(x), the first derivative of 
the tone scale curve, is constructed by Gaussian interpolation functions. Since each interval is interpolated independ- 
ently, the interpolation process from x 1 to x 2 will be discussed without loss of generality. If it is assumed that the width 
of influence at x A and x 2 is ^ and <y 2 , respectively. The interpolation function has the following form: 
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g(x) = A exp[- l l ] + B exp[- /' ] + C, 



2o, 



2a 



(2) 



70 



and the three unknown coefficients A t B, and C can be solved from the boundary conditions: 

n*J = + B exp[- ( ^"* 2) ] + C 



2ct 



f (x 2 ) = $(x 2 ) = 4 exp[- ( ^"* 2> ] + B + C 



2a; 



/5 



20 



The following quantities are defined as: 



25 



a = exp[ -^J 

2a | 



30 



(X« - 

2a 2 



35 



c = J exp[- ^— ]cfr 

2a, 



40 



* = J exp[- K -—lL-]dx 



2a 0 



Then the solution for A. B, and C can be expressed as: 



45 


-A- 




■1 


a 1 i 


-1 






B 




b 


1 1 




f(x 2 ) 




.C. 




_c 


d (x 2 * 




_(y 2 -ri). 



(3) 



so It is possible that the matrix can be singular and there exists no solution for the particular constraints. The algorithm 
then reduces a-, and <y 2 and tries to solve the equations again. This cycle will repeat several times until at and a 2 
become too small and the algorithm simply defaults to a constant slope solution for the interval. The preferred values 
of a at the three given points are a 1 = 100.0 for the shadow, a 2 = 300.0 for the mid-tone, and a 3 = 80.0 for the highlight. 
Figure 9 shows an example of the tone scale curve constructed by the method in this invention. The noisy curve at the 

55 bottom is the histogram of the input imaga 
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The Contrast Gain-Control (CGC^ Module 

[0029] A large dynamic range of a natural scene is almost always caused by illumination differences in various parts 
of the scene. The boundary between two different illuminations is called the illumination edge, which is often of high 

s contrast and high exposure gradient In comparison, the boundary between two different reflecting materials is called 
the reflectance edge, which is often of lower contrast and smaller exposure gradient. An ideal way to compress the 
dynamic range of an image is to reduce the magnitudes of the illumination edges, while preserving all reflectance 
edges. Figure 10 illustrates how this ideal compression operation should be performed. An important step to achieving 
such selective edge magnitude compression is to identify where the illumination edges are in the image. ©'rl Sg'trTeillu^ 

10 mination edges are"l6catidrthe"rnagnitude of the edges caTTbe controlled by adjusting the contrast g^irT-SMrSra^iiea 
totheiUumm edges 
inthe^nput ima ge and generate a gain-control signal for each p jx el of the im age. These^jain-cohtroi signals are applied 
tfrTheTSiter banks whose job is to extract edge signals of all~magnrtude. The small-magnitude reflectance ed ges are 
given a contras t gain of 1.0 or higher, while the large-amplitude illumination edges are given a very small gain. As a 

15 result of this gain control mechanism/the illumination edges are greatly suppressed in the output, D\ of the filter bante 
(see Figure 4). Ideally, D' = 0 at pixels where illumination edges occur. Therefore, the illumination edges in the final out- 
put image, t out , are griafn ly frc^^ version oflhelhput img ge, w hich is, of course, prope rly compressed in 
it s dynamic range . MaSematically, I out = /' + D' = /' for illumination edges. Iri comparison, the reflectance edges in the 
"aSference imag'e D pass through the filter banks virtually unattenuated. Therefore, the output D' from the filter banks 

20 module contains almost the exact copy of the reflectance edges in / - /', and when it is added to /' to produce f out , the 
reflectance edges are well preserved in the output image. Mathematically, I out - I' + D' =/' + (/-/') = / for reflectance 



[0030] One important fact in image formation is that illumination edges are not easily distinguishable from reflectance 
edges at a small spatial scale because certain reflectance edges also have large edge magnitudes when edge magni- 

25 tudes are computed from small spatial^ distances. A/eliable det ection of illumination edges has to be j>erformed over 
JargespafeLdistances. Tftjs is a very irnportanf point £ 
pitted Jrom a coarse s<^lej^ge d sfcheme that attempts to su^resVSie^airfting^artifacfe Has to use a 7 

coarse scale edge detector to control the c»nfrag[gain factors for edge'f ilterihg. This is why a gain-control function gen-^ 
erated irrterrmll v" within e ach TSs^ution can not work well. The gain-control signal for the fine resolution details has to 

30 cqme from the coarse resolution edges. <s'c>4/^ /Z{& 

[0031] Figure 1 1 shows the preferred embodiment of the contrast gain- control signal gener ation. T&e input i mage, is 
?ffi1*r^tlirou^ of th^fihe refi^tance edges. The lo w-p^^fjlterJL2 170 &>7t?L 

in"Rgure 11 is related to the f!Hers^tJi<^^ in'Figure 4^ In the pre- F*h, 

ferred embodiment, N = 6, both F 5 and F 6 have the same coarse resolution, the common low-pass filter in F 5 and F 6 

35 is S(© x )S(a> y )S(2o> x )S(2to y ) and the filter L should be chosen to have a comparable coarse resolution. A simple choice 
is to set L(G> xt G> y ) = S{& x )S(<& y )S(2<s> x )S(2io y ) . The filters F 5 , F 6 , and S in the preferred embodiment will be 
described in detail shortly in the context of the filter banks module. Figure 12 shows a series of smoothing filters that 
can be used for various image sizes. The filter is generated from a basic kernel of [1 ,4,6,4, 1 ] with multiple convolutions 
with itself. In order to preserve the zero frequency response, the area under each smoothing filter is normalized to 1 .0. 

40 For practical implementation of the invention, the exact shape of the smoothing filter is not important as long as it is 
smooth and compact For example, a truncated Gaussian filter is also quite acceptable. 

[0032] The algorithm then computes the ed ge gradi ent on the low-pass filtered image by the gradient amplitude proc- 
essor 180. The edge gradient amplitude iscorrputecPby takiog4hasquare root of the sum of the square of horizontal 
pixel difference and the square of vertical pixel differenc^Phe pixel dih erence is comp uted-over a distance proportional 

45 to the size of the smoothing filter. The com puted edge jgyadjent amelitoqe is then mapped through a function g n 190 and 

the output, V , ts used to modified tfiFgaFh-corr tr ol sigrtefs-4^rt?2, • ■. G N .The general functional form of ^ftis chosen If 06$ 6HAU. 
so that when the edge gradient amplitude is small, its output value ]/ is Tl) and when the edge gradient amplitude is ^ 0 ~ S*#tll 

large, V, is zero. The V value is used as a maskjmage so that the functions g^, g 2 9n (200a, 200b ... 200n) can 

adjust its gain spatially. When the edge gra3IenTarrplSude is large, V is zero and the gain-control functions, g h will 

so reduce the local gain factor G, to suppress the edge banding artifacts. The N additional input signals to the Contrast 

Gain-Control module are the output 0 1( D 2 , .... D N , from the decomposition filters F 1( F 2 F N . These input signals are 

first mapped through d^,d 2 , .... d N (210a, 210b „210n) to calculate their energy For example, a simple squaring func- 
tion, dj(x) = x 2 , can be used. These N energy signals are then used to produce their weighted linear combinations via 0 
an Nx N matrix 220. The N output signals, C n ,C 2 C N , from the matrix are further modified by functions g^g 2 CA W 

55 g Nt (200a, 200b ... 200n) under the influence of V, to produce the final gain-control signals Gi ,G 2 . ...» G N . The general , " ' 

form of the functions, g^,g 2 9n* (200a, 200b ... 200n) is similar to that of g 0 (190). That is, g t produce gain factors 

close to 1.0 when C, is small. However, when C, is large, the output of g t depends on V. If V is close to 1 .0, then G, is 
also close to 1 .0. If V is close to zero, then G, is suppressed to be near zero as well. 
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[0033] The operation of the CGC module can be better understood from the following example. The illumination edges 
usually have large gradient amplitudes at acoarse spatial resolution as deter mined by the low-pass filter L in Figure 1 1 . 

The value V is ten very smin^As-a-eensequence, all the gairvcontrol signals. Gi,G 2 G N , are also set at a very small 

value rf C 1( C 2 C N are large, and the illumination edge is suppressed. In comparison, reflectance edges are mostly 

5 smoothed out by the smoothing filter L (1 70) and only have small gradient amplitudes. Therefore, V will be close to 1 .0. 
As a consequence, all the gain-control signals, ,G 2 , ... G N , are also set at a value close to 1 .0. and the reflectance 

edges in D )t D 2 D N of Fig. 4 are passed without any reduction in magnitude. It is worth pointing out again that this 

capability to discriminate between the illumination edges from the reflectance edges comes from the low-pass filter, L 
(170), as well as the amplitude mask produced by g 0 (190). 

10 

The Filter Banks (FE\ Module 



75 



25 



[0034] The Filter Banks (FB) Module (see Figure 4) receives the difference image D = I - (', which represents the 
image details in theo ricrinal image /Jr^aLa*e4ostjathedy^ range compressed image /'. These details include both fftin rn'Ss 
the fine reflectance edges and thq^co arse ill umination ed ge^L ""^ffihn* "Itf^ banks is to extract the high frequency ^ 
details, and to remove the very lowTrequency (DC) component that constitutes the major portion of the large dynamic 
range in the input image. But, not all the high frequency details should be allowed to pass through the filter banks. The 
part of high frequency signal that is associated wrjjxthe illumination edges should be suppressed and the suppression 
is achieved by the gain-control signals from the/JG^Vnodule. 
20 [0035] The number of filter paths, A/, is determined by the cost and system throughput constraints. As the number of 

filter paths increase, the banding artifact reduces.-becai^e more frequency channels allow the illumination edges to be w 
suppressed more successfully as a function of ^patialscale^> Referring to Figure 4, one of the unique feature of the filter tyft 
banks in this invention is to separate each f irter paThlrrto-Kvo parts, F,- and H h with a nonlineaj^ain=£aaifeo! 4o between^^ ^ \j( 
This design is desirable because the filter 5 is designed to be edge (or texture) detectors ir^djfferent spatiaTscales aV td ' «rfft9 
orientations. Thererore tne trom filter /-, represents edge signals at a particular scale in a particulaT^p mldTi^e r Yte llo^ fll*" 
It is this explicit edge contrast signal that is modulated by the contrast gain-control signal. Since the optimal decompo- 
sition by Fj into edge representations does not necessarily lead to an optimal reconstruction by simple summation, a 
post processing filter. H h is required to reconstruct the difference image back. (Mathematically, F, filters are not orthog- 
onal.) There are many possible choices for the decomposition-reconstruction fitter pairs. The number of filter paths, A/, 
30 and the image size will constrain the choices to a few for an efficient implementation. In the preferred embodiment, three 
frequency bands and two spatial orientations (vertical and horizontal) are selected, making the total number of paths 
equal to N = 3 x 2 = 6. Let and <o y be the spatial angular frequency in the x and y directions. The filters have the 
following frequency 

35 F,(G> x ,<o y ) = R(g> x ) (4) 

H,(co x ^ y )=Q(a> x )L(m y ) (5) 
F 2 (© x .a> y )= fl(cD y ) (6) 

40 

H 2 (o> x ,a> y ) = L(<o x )Q(G> y ) (7) 
F 3 (a> x ,© y ) =S(<Q x )S{<o y )Ft(2<D x ) (8) 
<5 H 3 (a> xt a> y ) = P(<o x )P(a> y )Q(2a> x )L(2<o y ) (9) 

F A (G> x ,<i> y ) = S(a> x )S(G> y )R(2G> y ) (10) 
H 4 {<o x ,o> y ) = P(<o x )P(<o y )L(2<o x )Q(2G> y ) (11) 
F 5 (<o x ,<o y ) = S{a> x )S{<£> y )S(2 G > x )S(2<o y )R(Aa> x ) (12) 



50 



H 5 (© X( c f > y ;=P^ x )P(co y )P(2cD x )P(2G> / )0(4a> x )/.(4G) / ; (13) 
ss F 6 (© x ,<o y ; = Sr<o x )S(<o y )S(2a> x )S(2a> / )R(4G) x ) (14) 

H 6 (<o x ,ca y ;=Pr©JP(G> y )P(2<o x )P(2a) y )L(4(o x )0(4o) y ) (15) 
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where 



S(<o) = cos 4 (|) 



(16) 



H(a>)=/sin(|)e /<0/2 



07) 



P(co) = S(e>) 
Q(cd) = [1-S 2 (<o)]/R(o) 
L(<d) = [1 + S 2 (<o)]/2 



(18) 
(19) 
(20) 



These filters are from the family of edge-wavelet filters that were used in Mallat and S. Zhong's paper, "Characterization 
of signals from murtiscale edges," IEEE Transactions on Pattern Analysis and Machine Intelligence, 14, 7, 710-732, 
1992. The fitter banks can also be implemented in a different form as shown in Figure 13, where the simplified notations 
have been used to make the figure more readable. For example, SxSy(u>) denotes S(<a x )S(a> y ), Rx(2g>) denotes 
ff(2© x ), and so on. It should be pointed out that these filters are not the only choices. The preferred choice is that the 
Fj filters should extract edge-like information. 

[0036] From the definition of Fourier transform of a sampled function f(n): 



(21) 



f(n) = ± \ F(©)e*' n <fc>. 



(22) 



the fitter coefficients are derived as: 
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Producing the Output Image 

[0037] The output signal D' from the filter banks contains all the fine details that are in the original input image / , but 
are missing in the tone-scaled and dynamic-range-compressed image /'. One of thejr^UmpQrtarit features of this 
jnventjonjs ^that tii e highjx>n tostjllu^ suppressed by the corrtrasfgSrvcontroI andlrrerefore they~ are 

"not presentTrTir/the output ofthe filter banks module By addmg D'back to /'Tthe-algorithm produces an output image 
l ou t that is free from the banding artifacts in the dynamic range compressed image. Figure 14 shows a comparison of 
the input signal profile of a line through the input image / and the output signal profile of the same line through the output 
image l out . The dynamic range of the input image signal is greatly reduced and yet, there is no banding artifact at the 
very high contrast edges (around pixel locations 160 and 170). The low-contrast image details at other locations are 
well preserved. 
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Claims 



10 



45 



55 



1. A method of reducing the dynamic range of an input image for effectively rendering the input image on an output 
display medium, the method comprising the steps of: 

(a) extracting detail signals and coarse signals from the input image; 

(b) generating contrast gain-control signals from the input image by detecting coarse scale edges; 

(c) modifying the detail signals according to the contrast gain signals; and, 

(d) adding the modified detail signals to the coarse signals to obtain an output image. 

2. The method as in claim 1 , wherein step (b) comprises the steps of: 



(bl) processing the input image through a low-pass filter to obtain a blurred version of the input image; 
(b2) computing gradient amplitudes of the blurred image; and, 
is (b3) using the gradient amplitude as an index into a look-up table to generate the contrast gain control signal. 

3. The method as in claim 2, wherein the step of extracting the detail signals includes processing the input image 
through a plurality of band pass and high pass filters. 

20 4. The method as in claim 1 further comprising modifying the coarse signals with a tone scale curve after step (c). 

5. A method of reducing the dynamic range of an input image / so that it can be rendered effectively on an output dis- 
play medium, comprising the steps of: 

25 (a) constructing a tone scale curve 7 from the input image /; 

(b) mapping the input image / through a tone scale curve T to produce a tone-scaled image /' = T(l) ; 

(c) producing a difference image D = / - /' by subtracting the tone-scaled image from the input image; 

(d) producing a modified difference image. D\ by processing the difference image D through a plurality of 
band-pass and high-pass filters; and, 

30 (e) adding the modified difference image D' to the tone-scaled image /' to produce a dynamic range com- 

pressed output image. 

6. The method according to claim 5, wherein the step (a) of constructing a tone scale curve comprises the steps of: 

35 (a1) computing the histogram from the input image; 

(a2) finding the low end, the median point, and the high end of the input values from the said histogram at some 
fixed percentiles; 

(a3) mapping the low end point to the black level of the output medium with a higher than normal local slope; 
(a4) mapping the median point to the medium gray level of the output medium with a normal contrast; and, 
to (a5) mapping the high end point to the white level of the output medium with an adjustable contrast, depending 

on the shape and the range of the upper tail of the said histogram. 

7. The method according to claim 5, wherein the band-pass and high-pass fitters used in step (d) are edge detectors 
at various spatial scales. 



8. A method of generating a tone-scale curve from an image for dynamic range compression, comprising the steps of: 



(a) computing a histogram from the input image; 

(b) finding a low end, a median point, and a high end of the input values from the histogram at some fixed per- 
so centiles; 

(c) mapping the low end point to a black level of the output medium with a higher than normal local slope; 

(d) mapping the median point to a medium gray level of the output medium with a normal contrast; and, 

(e) mapping the high end point to a white level of the output medium with an adjustable contrast, depending on 
the shape and the range of an upper tail of the histogram. 



9. A method of image contrast enhancement using contrast gain-control signals for suppressing banding artifacts, 
comprising the steps of: 
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(a) decomposing an input image into edge gradients of several spatial scales; 

(b) generating contrast gain-control signals G from the original input image; 

(c) enhancing each edge gradient signal by modifying its amplitude according to separately generated contrast 
gain-control signals G in step (b); and, 

(d) reconstructing an enhanced output image from the modified edge signals of step (c). 

10. The method according to claim 9, wherein the step (b) of generating contrast gain-control signals G comprises the 
steps of: 

(b1) low-pass filtering the input image; 

(b2) computing an image gradient from the low-pass filtered image; and, 

(b3) mapping the magnitude of the image gradient through a lookup table which is calculated from a smooth 
monotonic function to produce the contrast gain-control signals G. 
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(54) Automatic tone adjustment by contrast gain-control on edges 



(57) A method of reducing the dynamic range of an 
input image for effectively rendering the input image on 
an output display medium, the method comprises the 
steps of extracting detail signals and coarse signals 



from the input image; generating contrast gain-control 
signals from the input image by detecting coarse scale 
edges; modifying the detail signals according to the con- 
trast gain signals; and, adding the modified detail sig- 
nals to the coarse signals to obtain an output image. 
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